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report  describes  a  concept  of  usinj;  realistic  radio  communications  activities 
as  subsidiary  workload  measurement  tasks  in  KM)  simulations  and  actual  flii’Jit 
tests.  In  order  to  devise  a  generalized  method  of  developing  communications 
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^Block  20.  Abstract  (continued) 

tasks  with  controlled  levels  of  added  workload  which  could  be  tailored  to 
specific  system  and  mission  contexts,  three  methods  of  workload  scaling 
were  applied  to  11  communications  tasks  typical  of  those  occurring  in 
the  A-10  aircraft.  The  first  technique  provided  workload  estimates  based 
on  the  information  transmission  demands  of  communications  activities. 
Information  theoretical  metrics  were  applied  to  the  perceptual  decisions 
and  manual  action  decisions  required  in  response  to  incoming  messages.  In 
a  second  scaling  effort,  this  approach  was  supplemented  by  estimates  of  the 
additional  contribution  to  workload  of  memory  demands,  information 
gathering  activities,  and  instruction  complexity,  which  were  not  as>.e  ,  ' 
by  the  information  theoretical  measures  jv.l’i  I  ot  opinions  ot  tin  wo,  :  . 

associated  with  messages  contained  in  tlW  cnninuinirut  ions  task-,  wet. 
obtained  bv  a  paired  comparisons  technique.  flu-  t  cults  the  initial 
analytical  scale  and  the  weightings  derived  I rom  this  procedure  were 
combined  to  lorm  a  hvhrid  analytical  seal. .  The  final  scaling  approach 
relied  on  subjective  estimates  ol  the  workload  associated  with  complete 
t ommnn ieat ions  tasks.  Pilot  rankings  of  the  tasks  were  used  to  derive  a 
scale  based  on  modified  Thurstonian  procedures.  Nonparamet r ic  correla¬ 
tional  procedures  revealed  considerable  agreement  among  the  results 
of  the  three  scaling  methods. 

The  final  section  of  this  report  outlines  a  plan  for  experimental  dual  task 
performance  studies  to  test  the  sensitivity  of  the  communications  tasks  to 
primary  task  workload  and  to  evaluate  the  three  a  priori  scaling  methods. 
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This  report  describes  the  development  and  scaling  of  pilot  radio  communica¬ 
tions  activities  for  use  as  operationally  oriented  subsidiary  workload 
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Section  1 


INTRODUCTION 


CONCKPTUAI.  BACKGROUND 

Mental  workload  is  a  problem  of  increasing  importance  in  modern  airborne 
weapon  systems.  The  expanding  capabilities  of  military  aircraft  achieved 
through  the  incorporation  of  sophisticated  technology  are  responsible  for  a 
proportional  growth  in  the  monitoring  and  decision  making  responsibilities 
of  the  individual  crew  member.  As  these  task  demands  continue  to  accel¬ 
erate,  human  information  processing  capacities  and  limitations  become  more 
critical  determinants  of  total  system  performance.  Consequently,  in  order 
to  insure  mission  success,  a  variety  of  accurate  and  reliable  methods  are 
needed  to  assess  aircrew  workload  at  all  levels  of  system  development. 

Because  mental  processes  are  not  directly  observable,  a  number  of  subjec¬ 
tive,  physiological,  and  performance  indices  have  been  offered  as  measures 
of  mental  workload.  Concerns  for  quantification  and  reliability  make 
objective  measures  of  workload  preferable  to  subjective  estimates  in  many 
applied  settings.  The  secondary  task  methodology  is  the  most  widely  used 
objective  measure  of  workload  and  is  the  technique  with  the  greatest  amount 
of  research  support  (Wierwille  and  Williges,  1978).  The  use  of  secondary 
tasks  is  based  on  the  assumptions  that  an  upper  bound  exists  on  the  ability 
of  the  human  operator  to  process  information  and  that  the  mental  resources 
which  form  this  limited  capacity  can  be  shared  among  tasks.  The  method¬ 
ology  requires  the  operator  to  perform  an  extra  task  along  with  a  primary 
task  of  interest.  Workload  measures  are  derived  by  comparing  single  and 
dual  task  performances. 

As  noted  by  Ogden,  Levine,  and  Kisner  (1979),  secondary  tasks  can  be  used 
to  estimate  workload  in  two  general  ways.  In  one  type  of  experimental 
situation,  an  additional  task  is  added  to  induce  stress.  The  purpose  of 
this  manipulation  is  to  increase  total  workload  in  order  to  improve  the 
sensitivity  of  primary  task  performance  measures.  The  more  traditional 
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application  of  the  secondary  task  is  to  derive  a  measure  of  spare  mental 
capacity.  In  this  case,  the  primary  task  receives  priority  while  the 
secondary  task  is  relegated  to  residual  processing  resources.  An  estimate 
of  primary  task  workload  is  made  by  assessing  the  magnitude  of  the  dif¬ 
ference  in  secondary  task  performance  between  the  single  task  and  the  dual 
task  conditions. 

The  workload  literature  documents  a  variety  of  tasks  that  have  been  used 
within  these  two  paradigms  with  varying  degrees  of  success.  In  order  to 
provide  some  guidance  for  the  selection  of  appropriate  tasks,  Knowles 
(1963)  listed  several  characteristics  which  secondary  tasks  should  have  in 
order  to  maximize  the  sensitivity  and  validity  of  measurement.'  Tasks 
should  be  easily  learned  and  scorable,  and  task  demand  should  be  manipu- 
lable  over  a  range  of  difficulty.  In  order  to  eliminate  peripheral  inter¬ 
ference,  secondary  tasks  should  not  share  input  or  output  modalities  with 
the  primary  task.  Furthermore,  secondary  tasks  which  tap  a  variety  of 
'nformation  processing  functions  are  preferable  to  those  which  load  only 
specific  cognitive  structures.  Ogden  et  al .  (1979)  added  the  important 
criterion  of  acceptance  by  the  operator.  Whether  used  to  induce  stress  or 
to  measure  reserve  capacity,  secondary  tasks  should  be  chosen  to  achieve 
face  validity  and  congruence  with  the  overall  performance  situation. 
Operator  acceptance  is  an  especially  crucial  factor  in  operational  environ¬ 
ments  where  failure  to  integrate  the  extra  task  would  lead  to  the  contami¬ 
nation  of  results  because  the  operator  either  neglects  the  task  or  allows 
it  to  assume  an  artificially  high  priority. 

Although  some  secondary  tasks  come  close  to  meeting  many  of  these  criteria 
in  a  given  situation,  additional  problems  arise  with  most  tasks  when  their 
use  at  different  stages  of  system  development  is  taken  into  consideration. 

The  criteria  discussed  by  Knowles  (1963)  are  specific  to  practical  issues 
associated  with  secondary  task  methodology.  Theoretical  considerations  may 
also  place  restrictions  on  task  selection  and  interpretation.  These  fac¬ 
tors  are  addressed  in  Section  V  of  this  report. 
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Schiflett  (1976)  noted  that  the  majority  of  workload  measures  were  devel¬ 
oped  for,  and  are  most  applicable  to,  the  early  design  stage  in  which 
experimentation  is  confined  to  a  laboratory  setting.  At  the  level  of 
operational  test  and  evaluation  or  even  during  high  fidelity  simulation, 
these  measures  often  become  difficult  or  impractical  to  employ.  Wierwille 
and  Williges  (1978)  reviewed  several  factors  which  affect  the  feasibility 
of  using  various  workload  measurement  techniques  in  an  in-flight  environ¬ 
ment.  Physical  variables  such  as  the  size,  weight,  and  portability  of 
experimental  equipment  are  obvious  problems  that  can  be  solved  with  further 
technological  development.  However,  the  potential  for  intrusion  on  primary 
flight  control  performance  is  a  danger  that  would  accompany  the  use  of 
nearly  all  traditional  secondary  tasks.  The  probability  of  such  inter¬ 
ference  would  be  greatest  when  the  added  task  is  artificial  and  novel  in 
nature  and,  therefore,  apt  to  cause  involuntary  distraction  from  critical 
act  ivities. 

The  combined  problems  of  operator  acceptance  and  intrusion  severely  limit 
the  use  of  objective  secondary  task  measures  of  workload  in  high  fidelity 
flight  simulation  or  operational  test  environments.  At  best,  presently 
available  laboratory  tasks  have  questionable  validity  in  these  situations, 
and  at  worst  could  impair  flight  safety.  For  these  reasons,  workload 
measurement  at  the  critical  later  stages  of  system  evolution  is  often 
performed  using  relatively  informal  and  qualitative  techniques. 

SUBSIDIARY  COMMUNICATIONS  TASKS 

A  secondary  task  which  would  be  suitable  for  assessing  mental  workload  or 
for  increasing  workload  stress  during  high  level  simulation  or  in-flight 
test  should  be  fully  integrated  with  existing  system  hardware  and  with  the 
crew  member's  conception  of  his  task  and  mission  environment.  By  its 
nature,  such  a  task  would  be  a  realistic  component  of  crew  station  activity, 
yet  one  which  is  logically  and  experimentally  separable  from  the  primary 
flight  performance  of  interest.  The  effort  described  in  this  report  was 
directed  toward  developing  methods  for  adapting  radio  communications  activi¬ 
ties  for  use  .is  secondary  loading  tasks  which  would  fulfill  1  these  criteria 
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while  providing  a  quantifiable  means  of  measuring  the  workload  of  aircrew 
members. 

The  aircraft  radio  communications  which  appear  to  be  most  amenable  to  this 
application  are  those  initiated  by  a  message  sent  from  anv  of  a  number  of 
combat  elements  to  the  pilot  whose  level  of  workload  is  to  be  assessed. 

Upon  detection  and  identification  of  a  relevant  message,  the  pilot  must 
engage  in  verbal  and  manual  actions  in  response  to  the  information  that  he 
receives.  Such  tasks  closely  resemble  the  nonadaptive  discrete  secondary 
tasks  used  in  numerous  workload  studies  (see  Wierwille  and  Williges,  lf*7S) 
and,  upon  further  analysis,  appear  to  embodv  many  properties  of  a  good 
workload  measure.  Initially,  it  is  apparent  that  communications  activities 
call  upon  a  wide  range  of  perceptual,  cognitive,  and  motor  abilities  and 
have  the  potential  of  being  varied  along  several  dimensions  of  difficulty. 
Furthermore,  the  opportunity  for  peripheral  input  or  output  interference  is 
minimized.  Communications  tasks  occupy  only  the  auditory  input  channel, 
thereby  eliminating  effects  of  sensory  interference  which  would  reduce  the 
validity  of  any  visual  secondary  task.  Likewise,  the  chance  for  output 
interference  is  obviated  since  verbal  responses  are  uniquely  assigned  to 
communications  functions  in  present  aircraft  systems  and  al 1  manual 
switching  activities  are  designed  to  be  dealt  with  by  the  pilot's  hand  not 
required  for  aircraft  control.  From  the  hardware  point  of  view,  communi¬ 
cations  tasks  are  advantageous  secondary  tasks  because  they  require  no 
additional  displays  or  controls  for  the  pilot;  and  they  permit  the  experi¬ 
menter  to  control  task  presentation  and  to  record  performance  data  using, 
existing  communications  channels. 

Most  importantly,  communications  tasks  are  already  an  integral  part  of  the 
pilot's  in-flight  duties.  As  a  result,  lengthy  training  recjui rements  are 
eliminated  and  high  face  validity  is  achieved.  Furthermore,  the  realistic 
nature  of  the  task  makes  artificial  task  interactions  and  intrusion  improb¬ 
able  because  the  pilot  has  predetermined  attentional  priorities  assigned  to 
both  communications  and  other  cockpit  functions. 


The  concept  of  using  communications  tasks  to  manipulate  workload  was  tested 
informally  during  a  recent  study  using  an  A-10  full-mission  simulation 
(Spencer,  197')).  While  the  pilots  delivered  stand-off  weapons  against  tank 
targets,  communications  tasks  requiring  radio  switching,  radio  tuning,  and 
verbal  responses  were  presented  in  order  to  increase  workload.  Although  no 
quantitative  data  were  presented,  Spencer  reported  that  the  effects  of 
communications  stressors  on  total  system  performance  were  operationally 
significant. 

These  suggestive  results,  combined  with  the  urgent  requirement  for  nonin- 
trusive  objective  measures,  stimulated  the  research  to  develop  and  validate 
the  concept  of  using  secondary  communicat ions  tasks  for  workload  assessment 
in  R&I)  simulation.  A  primary  goal  was  to  devise  communications  tasks 
quantified  with  respect  to  workload  and  which  would  maintain  a  high  level 
of  operational  realism.  Accordingly,  specific  aircraft  and  mission  types 
were  selected  to  derive  the  source  material  to  be  used  to  generate  communi¬ 
cations  tasks  and  to  validate  the  resulting  workload  assessment  method¬ 
ology.  The  formulation  of  secondary  communications  tasks  for  A-10  air-to- 
ground  missions  is  described  in  this  report.  Evaluation  of  these  secondary 
tasks  will  he  performed  in  limited  and  full  scale  mission  simulation.  If 
the  results  of  the  evaluation  suggest  that  this  approach  is  viable,  addi¬ 
tional  tasks  will  be  developed  to  fulfill  other  Air  Force  workload  assess¬ 
ment  requirements.  Care  will  be  taken  to  obtain  review  of  such  tasks  by 
potential  users  to  ensure  that  face  validity  and  pertinence  of  the  communi¬ 
cations  tasks  are  preserved. 

The  initial  phase  of  task  development  involved  the  acquisition  of  source 
materials  for  the  A-10  air-to-ground  scenarios.  Extensive  interviews  with 
a  current  Tactical  Air  Command  A-10  pilot  led  to  the  compilation  and  organi¬ 
zation  of  the  13  communications  tasks  shown  in  Appendix  1.  The  tasks  were 
drawn  from  six  scenarios  representative  of  the  types  of  communications 
which  would  occur  as  the  pilot  (Tiger  1)  leaves  his  holding  orbit  outside 
the  forward  edge  of  battle  area  (FEBA),  descends  to  terrain  avoidance  (TA) 


altitude,  penetrates  the  FEBA,  and  completes  the  attack  phase  of  the  air- 
to-ground  mission.  The  scenarios  include  identification,  threat  alert, 
traffic  control,  waypoint  passage,  jammed  communications ,  and  strike  clear¬ 
ance  activities.  Each  scenario  contains  2  to  3  communications  tasks.  A 
task  is  defined  as  an  entire  logical  sequence  of  verbal  and  manual  activi¬ 
ties  initiated  when  the  pilot  receives  a  radio  message  from  some  other 
combat  element.  The  combat  elements  contained  in  the  source  material  are 
identified  below. 


Combat  Elements 

Radio/Freq . 

Comments 

TIGER  2 

FM,  40.80 

Another  A-10  following 
in  trail  by  30  seconds 

NAIL  A 

UHF,  Channel  1 

Airborne  Forward  Air 
Controller  (FAC) 

PARADISE 

VHF,  122.1 

Airborne  Warning  and 
Control  System  (AWACS) 
Aircraf  t 

POUNDER 

VHF,  142.7 

Army  Strike  Team 
Commander 

DOGBONE 

UHF,  Channel  5 

Another  Army  Strike 
Team  Commander 

FRIENDLY 

Unknown  until 
given 

Another  AWACS 

Aircraf  t 

The  manual  responses  required  by  the  communications  tasks  involve  the  use 
of  the  UHF,  VHF,  FM,  IFF,  intercom,  and  transmit  controls  of  the  A-10  radio 
system.  Appendix  2  contains  an  illustration  of  the  relevant  A-10  control 
panels  and  lists  the  switch  positions  and  standard  procedures  relevant  to 
the  start  of  a  mission. 

THE  SCALING  PROBLEM 

As  noted  previously,  radio  communicat ions  tasks  such  as  those  identified 
for  the  A-10  air-to-ground  mission  have  many  potential  advantages  over 
standard  laboratory  secondary  tasks.  Among  the  most  important  of  these  are 


the  fidelity  and  operational  orientation  of  the  communications  tasks. 

These  features  should  improve  acceptance  by  professional  pilots  and 
decrease  the  possibility  of  irrelevant  interference  with  the  behavior  of 
interest.  However,  a  serious  disadvantage  of  the  proposed  methodology  is 
that  the  very  realism  of  communications  activites  makes  precise  experimen¬ 
tal  control  or  higher  order  scaling  of  task  workload  difficult. 

Traditional  laboratory  secondary  tasks  are  designed  to  impose  constrained 
and  highly  describable  stimulus  and  response  demands  upon  the  performer. 

The  parameters  of  such  tasks  are  easily  varied  to  manipulate  task  loading 
in  a  precise  fashion.  Normally,  these  tasks  contain  abstract  stimulus 
elements,  and  the  effects  of  prior  experience  can  be  minimized  to  permit 
control  of  expectancies.  In  addition,  secondary  tasks  often  are  selected 
so  that  they  can  be  presented  repeatedly  to  obtain  a  precise  level  of 
loading  during  an  experimental  session. 

In  comparison,  realistic  radio  communications  activites  are  complex  infor¬ 
mation  processing  tasks  which  vary  along  many  more  loading  dimensions  than 
typical  laboratory  tasks.  Linguistic  stimuli  are  relatively  unstructured, 
and  complex  verbal  and  manual  response  sequences  are  difficult  to  vary  in  a 
controlled  manner.  Furthermore,  any  attempt  to  use  repeated  presentation 
of  single  tasks  or  to  cons  train  the  constituents  of  stimuli  and  responses 
would  detract  from  the  face  validity  of  secondary  communications  tasks. 

Whether  used  to  generate  workload  stress  or  to  measure  reserve  capacity, 
the  value  of  the  communications  task  methodology  would  be  compromised 
unless  a  valid  estimate  of  the  workload  associated  with  each  task  can  be 
obtained.  Assuming  that  an  ordinal  or,  perhaps,  interval  scale  could  be 
derived  for  the  workload  associated  with  communications  tasks,  it  would  be 
possible  to  combine  tasks  realistically  so  that  controlled  levels  of  addi¬ 
tional  loading  could  be  produced  to  meet  the  needs  of  specific  simulations 
or  flight  tests. 
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Several  methods  of  scaling  can  he  applied  to  communications  tasks  to  provide 
quantified  workload  estimates  which  do  not  require  laboratory  performance 
testing.  Although  all  of  these  methods  are  based  on  sound  theoretical 
foundations,  each  has  unique  sources  of  error  associated  with  it  and  may 
not  provide  a  completely  adequate  assessment  of  workload.  Consequently,  in 
order  to  maximize  the  probability  of  obtaining  valid  workload  estimates, 
multiple  scaling  techniques  should  be  used.  The  following  sections  of  this 
report  document  the  use  of  three  scaling  techniques  which  rely  on  analyti¬ 
cal,  subjective,  and  hybrid  methods  of  workload  estimation  (see  Table  1). 

The  object  of  these  efforts  was  to  generate  alternative  a  priori  methods  of 
communications  task  workload  scaling  which  could  be  used  to  design  workload 
assessment  methodologies  tailored  to  specific  systems  and  missions.  The 
criterion  for  comparative  evaluation  of  these  techniques  will  be  derived 
from  the  results  of  performance  based  tests  of  the  communications  tasks 
which  will  be  performed  during  a  later  phase  of  this  project.  It  should  be 
noted  that  although  the  A-10  mission  source  tasks  were  the  focus  of  these 
efforts,  the  scaling  methods  were  designed  to  be  applicable  to  other 
mission  and  system  contexts. 

TABLE  1.  APPROACHES  TO  THE  DEVELOPMENT  OE  A 
COMMUNICATIONS  WORKLOAD  SCALE 

1.  ANALYTICAL  SCALING 

Information  Theoretic  Analysis  of 
Information  Transmission  Requirements 

II.  HYBRID  SCALING 

Supplementation  of  Estimates  of  Information  Transmission 
Requirements  by  Measures  of  Additional  Information 
Processing  Requirements 

III.  SUBJECTIVE  SCALING 

Pilot  Workload  Ratings  and  Categorization 


II 


Section  2 


ANALYTICAL.  SCALING 


INFORMATION  THEORY 

Analytical  methods  of  estimating  workload  rely  upon  the  postulates  of 
models  or  theories  of  human  perceptual,  cognitive,  and  motor  processes  in 
order  to  quantify  the  information  processing  demands  of  a  task.  The  analyti 
cal  technique  used  in  this  scaling  effort  was  based  on  the  theory  of  informa 
tion  (Shannon  and  Weaver,  1949).  Briefly,  information  theory  defines  the 
transmission  of  information  as  the  resolution  of  uncertainty  concerning  a 
set  of  events.  In  general,  uncertainty  is  a  logarithmic  function  of  the 
number  of  alternative  events  that  could  occur.  The  information  metric  is 
the  hit  and  refers  to  the  number  of  binary  decisions  required  to  resolve 
uncertainty.  Applied  to  the  human  operator,  information  theory  offers  a 
method  of  quantifying  the  information  content  of  perceptual -motor  tasks. 
Considerable  experimental  support  is  available  to  indicate  that  information 
processing  time  is  a  roughly  linear  function  of  the  information  trans¬ 
mission  demand  of  a  variety  of  tasks  (see  Fitts  and  Posner,  1967).  The 
assumption  made  in  the  following  analysis  is  that  workload  is  partially  a 
function  of  the  uncertainty  associated  with  communications  task  stimuli  and 
responses . 


ANALYSIS 


The  workload  of  any  complex  task  can  be  considered  a  joint  function  of  the 
rate  of  information  presentation  and  of  the  complexity  of  the  information 
processing  required  of  the  operator.  Within  the  limits  of  realism  set  by 
the  simulated  missions,  the  rate  of  communications  task  presentation  will 
be  relatively  easy  to  control  and  should  not  prove  to  be  a  significant 
problem  when  used  to  manipulate  workload.  The  difficulty  of  scaling  the 
workload  of  communications  tasks  stems  from  the  multidimensional  nature  of 
the  complexity  parameter.  From  the  operator's  viewpoint,  complexity 
depends  upon  the  information  content  of  the  messages  received  and  upon  the 
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nature  of  the  cognitive  activity  required  to  translate  incoming  information 
to  an  appropriate  set  of  responses.  In  order  to  quantify  these  psycho¬ 
logical  factors,  at  least  two  analytical  approaches  are  required.  The 
communications  activities  first  must  be  ordered  in  terms  of  the  amount  of 
information  contained  in  messages  received  and  responses  produced. 

Secondly,  the  scaling  arrived  at  through  the  first  approach  must  be 
weighted  by  obvious  cognitive  activities  which  contribute  to  the  difficulty 
of  processing.  The  primary  purpose  of  the  scaling  effort  reported  here  was 
to  estimate  the  contribution  of  information  transmission  demands  to  the 
workload  of  communications  activities. 

The  generic  form  of  the  initial  communications  request  for  each  of  the 
tasks  found  in  the  source  material  is  as  follows: 


(1) 

Identif ication 

Of 

Intended  Receiver 

•  ft  •  1 

"Tiger  1" 

(2) 

Identification 

of 

Cal ler 

^  •  ft  •  * 

"Farad  iso" 

(3) 

Instruction 

"Squawk  Ident 

Although  each  of  these  steps  is  not  explicitly  stated  in  all  of  the  tasks, 
every  communication  requires  the  pilot  to  process  information  corresponding 
to  all  three.  Resulting  responses  include  both  verbal  and  manual 
activities. 

For  the  purposes  of  this  analysis,  the  pilot's  task  can  he  broken  down  into 
a  series  of  information  processing  decisions  related  to  the  three  steps 
shown  above.  The  number  of  processing  decisions  required  by  a  particular 
task  should  be  a  measure  of  the  contribution  of  information  transmission 
demands  to  workload.  As  dictated  by  formal  information  theory,  information 
is  transmitted  when  uncertainty  is  resolved  concerning  the  identification 
and  response  activities  contained  in  an  incoming  message.  Hit  values  (II) 
can  be  calculated  for  the  activities  required  of  the  pilots  in  communica¬ 
tions  tasks  by  estimating  the  number  of  alternatives  (N)  associated  with 
each  processing  decision  (ii  =  log  N)  . 
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Two  general  types  of  decisions  must  be  made  by  the  pilot  (see  Table  2). 

Once  lie  has  detected  the  presence  of  an  incoming  message  on  any  of  his 
monitored  frequencies,  he  must  make  two  perceptual  decisions.  First,  he 
must  determine  whether  the  message  is  intended  for  him  or  some  other  combat 
element.  Assuming  equal  probability  of  al ternatives,  there  are  six  persons 
who  could  be  called.  Thus,  the  resolution  of  the  uncertainty  associated 
with  this  decision  would  result  in  the  transmission  of  logv  6  =  2.58  bits. 
In  order  to  select  an  appropriate  transmitting  mode  and  frequency,  the 
pilot  must  also  identify  the  message's  sender.  Again,  this  would  result  in 
the  transmission  of  an  additional  2.58  bits. 


TAB  Li!  2.  COMMUNICATIONS  PROCESSING  DECISIONS 


I.  PERCEPTUAL  DECISIONS 

A. 

Identification  of 

Intended  Receiver 

B. 

Identification  of 

Cal  1 er 

11.  ACTION  DECISIONS 

A. 

Manual 

B. 

Verbal 

!■  ;i!  1  ,iw  i  up.  these  peiceptual  decisions,  the  pilot  must  make  ac  t  i  on  dec  i  s  i  oils 
;  i  i  ’spoii'ii*  t,i  the  instructions  he  receives.  Action  decisions  may  entail 
iii.iiiu.il  or  vernal  responses.  Manual  communications  activities  involve 
•iwi  I  oh  act  ions  on  the  IFF,  Intercom,  UIIF,  VHF,  and  FM  panels,  and  on  the 
!0  -.witcli  located  on  the  throttle  control  of  the  A— 10  (see  Appendix  2). 

Hu’  i  nt  oriii.it  ion  transmission  requirements  of  these  actions  can  he  quanti¬ 
fied  hv  investigating  the  sequence  of  manual  responses  and  the  number  of 
iltcin.il  ivu  actions  possible  for  each  response  designated  by  an  explicit  or 
i iied  command.  In  performing  tiiese  calculations,  equ  i  probab  i  1  i  ty  and 
independence  of  manual  decisions  were  assumed  as  well  as  complete  knowledge 
control  location.  As  an  example  of  these  computations,  consider  the  IFF 
panel.  Relevant  instructions  mav  require  a  single  push  of  the  I  DENT  switch 


or  could  involve  changing  IFF  codes  on  the  thumbwheel  controls.  Pushing 
IDENT  involves  the  transmission  of  a  single  bit.  However,  when  a  code 
change  is  required,  the  pilot  must  first  select  the  appropriate  mode 
(1  bit),  then  dial-in  a  four-digit  code  (12  bits)  on  the  thumbwheels  (if 
mode  3A  is  chosen),  and  finally  push  IDENT  (1  bit).  A  total  of  14.00  bits 
is  transmitted  in  this  sequence  of  activities  (see  Table  3). 

The  pilot  is  also  required  to  perform  manual  switch  actions  to  engage  in 
voice  communications  in  response  to  an  implied  instruction  (i.e.,  identi¬ 
fying  sender  and  switching  to  his  frequency)  or  to  an  explicit  instruction 
to  call  another  combat  element.  The  general  sequence  of  activitv  on  tin- 
communications  panels  is  shown  below. 


V11F/FM 

Manual 
On  1  v 

(4  Knobs) 


MIC 
(VII F) 


MODE  SELECT 


UHF 


VH  F /AM 


Select  Tuning 
Mode 


Manual 
(5  Knobs) 

MIC 

(UHF) 


Preset 

(20  Positions) 

MIC 

(UHF) 


Manual 

Only 

(6  Knobs) 


MIC 

(VHF) 


The  number  of  alternative  actions  for  each  switch  activitv  was  calculated 
by  inspection  (if  the  panels  and  from  knowledge  of  allowable  settings  on 
each  panel.  The  information  metrics  calculated  for  each  manual  response 
are  shown  in  Table  3. 


TABLK  3.  INFORMATION  THKORKT ICAL  CALCULATIONS 


Perceptual  Decisions 

A.  Possible  Senders  (6)  (  2.585  bits) 


l. 

Tiger  2 

2  • 

Nail  4 

3. 

Paradise 

4. 

Pounder 

5. 

Dogbone 

6. 

Friendly 

B.  Possible  Receivers  (b)  (  2.585  bits) 

Tiger  1  plus  all  of  the  above  minus  1 


Action  Decisions 

A.  Simple  Single  Switch  Action  (Ident)  (2)  (  1.000  bit) 

1.  Yes 

2.  No 


B.  IKK  Code 

1 .  Mode  l 

2.  Mode  3A 

If  mode  l:  two  dials  with  8  and  A 
alternatives,  respectively 

If  mode  3A:  four  dials  with  8 
alternatives  each 


(  1.000  bit) 


(  5.000  bits) 
(12.000  bits) 


C.  Select  Comm  Mode  (3)  (  1.585  bits) 

1 .  VHK  AM 

2.  VHK  KM 

3.  UHK 


1).  Mic  Switch  (2) 

1.  Up  (VHP) 

2.  Down  (UHK) 


(  1.000  bit) 
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TABLE  3.  INFORMATION  THEORETICAL  CALCULATIONS  (continued) 


t 

i 

i 

j 


E. 

F. 

Select  Preset  UHF  Channel 

Twenty  channels 

Release  Chaff  and  Flares 

(  4.322 

bits) 

1.  Chaff  (2) 

a.  Yes 

b .  No 

(  1.000 

bit) 

2.  Flares  (2) 

a.  Yes 

b.  No 

(  1.000 

bit) 

C. 

H. 

Select  VHF  Frequency 

Six  dials  with  0,  5,  10,  10,  4,  and 

2  alternatives,  respectively 

Manual  UHF  Channel  Select 

(11.966 

bits) 

1.  Select  Mode 

a.  Preset 

b.  Manual 

c.  Guard 

(  1.S85 

bits) 

2.  Select  frequency 

Five  dials  with  2,  10,  10,  10, 

and  4  alternatives,  respectively 

(12.966 

bits) 

t. 

Manual  FM  Channel  Select 

Four  dials  with  b,  10,  10, 

and  4  alternatives,  respectively 

(11 .229 

bits) 

.1 . 

Jammed  Comm 

1.  Chant’e  channels 

2.  Stay  on  same  channel 

(  1.000 

bit) 
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The  second  type  of  action  decision  which  is  required  by  an  instruction 
involves  verbal  responses.  An  overview  of  the  verbal  activities  contained 
in  the  source  material  revealed  two  general  types  of  responses.  The  sim¬ 
plest  activity  is  a  confirmation  of  a  message  or  switch  action.  This 
activity  may  be  confined  to  a  single  word  of  acknowledgement  ("ROGER")  or 
may  involve  the  repetition  of  an  instruction.  In  either  case,  the  task 
involves  pure  information  conservation  and,  at  this  level  of  analysis, 
requires  the  processing  of  a  single  unit  of  information.  The  second  type 
of  verbal  response  requires  the  pilot  to  select  a  receiver  (2.585  bits)  and 
report  a  specific  piece  of  mission-related  information.  In  some  cases 
reporting  is  immediate,  while  in  others  there  is  a  time  delay  between  the 
original  instruction  and  the  response.  Furthermore,  there  are  obvious 
differences  in  the  workload  associated  with  gatiiering  the  information 
content  of  the  report.  These  additional  factors  influencing  workload 
cannot  be  assessed  by  the  information  metric  since  it  is  limited  by  the 
assumption  of  complete  knowledge  of  report  content.  Therefore,  only  a 
single  bit  was  added  to  the  calculation  for  the  loading  attributable  to  the 
reporting  activity. 

The  calculations  of  the  information  transmission  requirements  for  eacli  of 
the  tasks  in  all  scenarios  are  shown  in  Appendix  3. 

A  number  of  limitations  to  the  analytical  approach  used  to  scale  loading 
must  he  recognized.  First,  somewhat  tenuous  assumptions  were  made  con¬ 
cerning  the  independence  of  sequential  activities  and  the  probability 
distributions  of  alternative  actions.  Second,  the  information  metric  could 
not  be  rigorously  applied  to  verbal  activities  because  of  the  inability  to 
define*  the  level  of  uncertainty  associated  with  speech  communication. 
Finally,  as  noted  earlier,  even  under  ideal  conditions,  measures  of  informa¬ 
tion  transmission  requirements  do  not  account  for  the  workload  contributed 
by  ttu'  nn  t  ii  re  of  the  information  processing  activities  necessarv  to  transmit 
information.  Variables  which  require  further  consideration  include  running 
memory  demands  which  play  a  role  in  scenarios  such  as  wavpoint  passage,  and 
the  loading  associated  with  the  specific  information  gathering  activities 
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reflected  in  the  content  of  verbal  report  responses.  Despite  the  limita¬ 
tions,  this  initial  attempt  at  scaling  produced  a  wide  range  of  workload 
estimates  when  applied  to  the  communications  tasks  taken  from  the  source 
material.  Table  4  summarizes  the  results  of  the  analysis. 
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IDENTIFICATION  DEMAND 


Section  3 

SUPPLEMENTARY  MEASURES  OK  THE 
INFORMATION  PROCESSING  DEMANDS  OF 
COMMUNICATIONS  REQUESTS 


PROCESSING  COMPLEXITY 

The  information  theoretical  measures  used  in  the  initial  attempt  to  scale 
communications  workload  provide  a  basic  index  of  the  information  transmis¬ 
sion  requirements  of  manual  and  verbal  tasks.  However,  such  measures  do 
not  account  for  the  contribution  to  workload  of  the  amount  and  complexity 
of  the  processing  activity  needed  to  produce  acceptable  performance.  The 
problem  is  illustrated  in  past  attempts  to  apply  information  theory  to 
choice  reaction  time  tasks.  Early  findings  suggested  that  the  time 
required  to  produce  a  response  was  a  simple  increasing  linear  function  of 
the  information  transmitted  (nyman,  1953).  However,  later  research 
revealed  that  such  factors  as  learning,  stimulus-response  compatibility, 
and  the  size  and  nature  of  transformations  on  stimulus  information  could 
vary  the  slope  of  this  function  dramatically  (Fitts  and  Posner,  1967). 
Similarly,  complex  tasks  sucii  as  those  performed  in  a  tactical  communica¬ 
tions  scenario  have  workloads  which  are  controlled  by  both  stimulus  and 
response  information  and  the  amount  and  tvpe  of  processing  necessary  to 
convert  stimulus  information  to  an  appropriate  response. 

The  purpose  of  the  effort  described  in  this  report  was  to  develop  a  method 
of  estimating  the  additional  contributions  to  workload  of  information 
gathering  activities,  memory  demands,  and  instruction  complexity,  not 
assessed  by  information  theoretical  measures.  An  examination  of  the  com¬ 
munication  activities  in  Appendix  1  revealed  three  general  factors  which 
may  add  to  the  workload  associated  with  communications  tasks.  First,  in 
some  of  the  tasks,  the  pilot  is  required  to  engage  only  in  specific  manual 
switch  actions  or  limited  verbal  behavior.  However,  other  instructions 
demand  that  he  perform  aircraft  control  maneuvers  or  gather  information 
from  cockpit  displays  and  from  the  external  environment.  Second,  many  of 
the  tasks  require  the  pilot  to  retain  information  in  morn orv  and  to  provide 
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TABLE  6.  DIMENSIONAL  ANALYSIS  OF  MESSAGES 


Message 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 


Memory 

Demand 


Information  Gathering 

Inside  Outside  Number  of 

Cockpit  Cockpit  Instructions 


1 

1 

2 

3 

4 
1 
i 

I 

1 

1 

I 

1 

1 

1 

1 


Inspection  of  this  table  indicates  that  specific  messages  vary  along  each 
dimension  and  that  while  some  of  the  messages  require  little  extra  pro¬ 
cessing  activity,  others  appear  to  demand  a  considerable  investment  of 
mental  resources. 

Because  of  the  multidimensional  nature  of  the  workload  associated  with 
these  messages  and  the  difficulty  of  quantitatively  specifying  the  loading 
induced  by  each  task,  a  structured  subjective  approach  was  taken  to  esti¬ 
mate  overall  information  processing  workload. 
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A  pa i roil  comparisons  technique  was  designed  to  obtain  pilot  opinions  of  the 

workload  associated  with  the  15  messages  extracted  from  the  communications 

tasks.  Kach  of  the  messages  was  paired  with  each  of  the  others  to  yield 
N (N-  1 ) 

or  105  comparisons.  The  paired  messages  were  randomly  ordered  and 
arranged  in  booklet  form.  During  test  administration,  32  current  A-7  and 
A-10  pilots  were  asked  to  examine  each  pair  and  to  indicate  witli  a  check 
mark  which  of  the  two  messages  entails  the  higher  workload.  Appendix  3 
contains  the  instructions  for  the  paired  comparisons  evaluation  and  a 
sample  page  from  the  test  booklet. 
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The  data  were  subjected  to  the  analysis  described  by  Normally  (pp.  51-55, 
1DP7)  to  derive  an  interval  scale  of  workload  based  on  Thurstone's  (1D27) 
haw  of  Comparative  Judgment.  Accordingly,  a  cross  matrix  was  formed 
showing  the  proportions  of  the  pilots  who  rated  each  message  greater  in 
workload  than  each  of  tire  others  (see  Table  7).  The  technique  required  the 
assumption  that  each  message  would  be  judged  greater  than  itself  half  of 
tire  time,  so  that  .5  is  placed  along  the  diagonal  of  the  table.  The  pro¬ 
portions  were  then  converted  to  normal  deviates  (Z  scores). 

Assuming  that  judgment  error  is  normally  distributed  about  the  "true" 
workload  scale  value,  Thurstone's  arguments  demonstrate  that  the  normal 
deviates  serve  as  the  interval  separating  the  workloads  of  two  messages. 

The  error  in  tire  normal  deviate  between  any  two  messages  is  reduced  by 
summing  the  Z  scores  in  each  column  and  deriving  the  mean.  The  resulting 
value  for  each  message  is  the  normal  deviate  expressed  about  the  average 
workload  in  the  set.  To  eliminate  negative  values  from  the  final  scale, 
the  absolute  value  of  the  largest  negative  value  was  added  to  each  of  the 
scale  values.  The  resulting  scale  is  shown  in  Table  8.  The  unit  of 
measurement  can  be  interpreted  as  one  standard  deviation  of  perceived 
difference  in  workload.  An  informal  interpretation  of  these  data  with 


TABLE  7.  PROPORTIONS  OF  PILOTS  WHO  RATED  MESSAGES  IN  COLUMNS 
HIGHER  IN  WORKLOAD  THAN  MESSAGES  IN  ROWS  (N  =  32) 


^See  Table 


TABLE  8.  DERIVED  WORKLOAD  SCALK  VALUES  FOR 
MESSAGES  EXTRACTED  FROM  A- 10 
COMMUNICATIONS  TASKS 


Message  Number* 

Value 

1 

.837 

0 

.493 

3 

.855 

4 

1.249 

5 

1.372 

6 

.909 

7 

1.192 

8 

.032 

9 

.354 

10 

.330 

1 1 

.284 

12 

0 

13 

.625 

14 

.229 

15 

.923 

*See  Table  5 


respect  to  ttie  four  dimensions  of  processing  listed  in  Table  6  was 
accomplished  by  computing  mean  scale  values  for  each  factor.  Messages 
containing  memory  demands  had  a  mean  workload  scale  value  of  1.180  in 
comparison  to  .481  for  those  messages  without  a  memory  component.  Instruc¬ 
tions  requiring  the  acquisition  of  information  from  the  visual  environment 
outside  the  cockpit  received  a  mean  value  of  .162  while  those  requiring 
information  gathering  from  visual  displays  in  the  cockpit  had  a  mean  value 
of  .987.  In  contrast,  messages  lacking  information  gathering  demands 
averaged  only  .444.  This  finding  is  of  special  interest  since  it  might 
have  been  expected  that  acquiring  information  from  a  complex  visual  scene 
would  present  a  greater  workload  than  obtaining  information  from  structured 


26 


displays.  Instead,  it  appears  that  perceived  workload  is  reduced  when 
pilots  are  able  to  use  concrete  information  from  the  "real  world"  to  ful¬ 
fill  the  demands  of  radio  communications  requests. 

An  initial  inspection  of  the  mean  scale  values  for  messages  containing 
differing  numbers  of  instructions  also  revealed  a  clear  trend.  Messages 
with  only  one  instruction  had  a  mean  value  of  .456.  Those  with  two  instruc- 
t  ions  averaged  1.023  on  the  scale,  and  those  with  three  or  four  instruc¬ 
tions  received  values  of  1.249  and  1.372,  respectively. 

The  informal  comparisons  discussed  above  are  of  limited  value  since  the 
dimensions  of  complexity  are  overlapped  in  many  of  the  messages.  In  order 
to  estimate  the  contributions  of  each  extra  processing  factor,  a  simple 
linear  additivity  model  of  workload  was  adopted.  That  is,  it  was  assumed 
that  combinations  of  these  dimensions  in  specific  messages  do  not  interact 
in  unique  ways  and  that  a  given  dimension  will  impose  tin1  same  degree  of 
workload  regardless  of  its  combination  with  other  dimensions.  The  analysis 
was  performed  by  summing  t he  scale  values  of  the  messages  which  contained 
each  of  the  seven  factors  contributing  to  workload.  The  dimensions  then 
were  used  to  generate  linear  equations  in  which  the  summed  effects  of  each 
relevant  factor  were  equal  to  the  total  scale  value.  For  example,  the 
memory  dimension  is  present  in  four  messages  (4,  5,  h,  and  7).  The  summed 
workload  scale  values  for  these  messages  is  4.722.  This  sum  is  a  result  of 
the  combination  of  the  effects  of  the  four  memory  components  and  of  the 
other  factors  operating  in  the  messages. 

Four  of  the  commands  with  memory  also  required  information  acquisition 
inside  the  cockpit  and  two  required  "heads-up"  visual  activity.  In  addi¬ 
tion,  each  instruction  length  was  represented  in  the  four  relevant 
mess.iges.  Thus,  the  linear  equation  derived  for  the  memorv  factor  was: 

4.722  =  4 ( M )  +  4 (IS)  +  2 (OS)  r  N  +  N  +  N  +  N. 


where 


M  =  memory  demand 

IS  =  inside  cockpit  information 

OS  =  outside  cockpit  information 

N;  =  one  instruction 

N  =  two  instructions 

N  =  three  instructions 

N,  =  four  instructions 

Seven  simultaneous  equations  were  generated  in  this  manner  based  on  the 
summed  scaled  values  for  each  dimension.  The  solution  for  the  equations 
yielded  normalized  scale  values  representing  the  workload  contribution  of 
each  dimension  (see  Table  9). 

TABLE  9.  NORMALIZED  WORKLOAD  WEIGHTINGS  FOR  ADD  IT IONA  I . 

INFORMATION  PROCESSING  DIMENSIONS 


Memory  Demands 

.140 

Gathering  Information  from 

Cockpit  Displays 

.299 

Gathering  Information  from 

External  Environment 

0 

One  Instruction 

.386 

Two  Instructions 

.  193 

Three  Instructions 

.609 

Four  Instructions 

.7  32 

The  values  obtained  by  this  procedure  were  then  used  to  modify  the  analyti¬ 
cal  scale  derived  in  the  preceding  section  of  this  report.  A  hybrid 
analytical  scale  representing  both  the  information  transmission  demands  and 
the  information  processing  complexity  of  the  source  radio  communications 
tasks  was  obtained  by  transforming  the  bit  value  obtained  for  each  task  to 
its  normal  deviate.  The  Z  scores  were  then  converted  to  eliminate  negative 
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values.  Finally,  the  wei.hts  shown  in  Table  9  were  added  to  the  normalized 
task  values  as  dictated  by  the  presence  of  each  added  processing  dimension 
in  the  communications  task.  The  resulting  scale  is  shown  in  Table  10. 


TABLE  10.  CALCULATION  OF  HVBKI1J  ANALYTICAL  SCALE  VALUES 


Section  4 

SUBJECTIVE  SCALING 


PILOT  RANKING 

Subjective  methods  of  workload  measurement  are  based  on  the  assumption  that 
the  judged  magnitude  of  the  conscious  experience  of  mental  effort  is  at 
least  partially  related  to  the  amount  of  information  processing  capacity 
required  by  the  performance  of  a  task.  Models  of  workload  which  emphasize 
the  relationship  between  physiological  arousal  and  loading  lend  indirect 
support  to  this  assumption  (e.g.,  Kahneman,  1973).  Unfortunately,  subjec¬ 
tive  methods  are  often  confounded  by  factors  such  as  emotional  state, 
experience,  skill  level,  and  simultaneous  physical  work.  Although  these 
sources  of  error  are  inherent  to  all  subjective  procedures,  the  experience 
of  effort  expenditure  commonly  reported  during  cognitive  processing  activi¬ 
ties  makes  it  reasonable  to  assume  that  structured  subjective  opinions  may 
provide  a  valid  estimate  of  the  workload  associated  with  radio  communica¬ 
tions  tasks. 

The  method  that  was  used  to  obtain  quantifiable  subjective  evaluations  of 
the  workload  of  the  source  A-10  communications  tasks  is  described  in  this; 
section.  A  scale  based  on  pilot  rankings  of  the  workload  of  complete 
communications  tasks  was  used  for  comparison  to  the  hybrid  and  analytical 
scales  obtained  in  the  previous  scaling  efforts. 


METHOD 

Thirty-one  A- 7  and  A-10  pilots  were  briefed  on  a  hypothetical  air-to-ground 
attack  mission  scenario.  The  briefing  identified  the  code  names  for  vari¬ 
ous  combat  elements  contained  in  the  source  tasks  and  outlined  standard 
procedures  relevant  to  communications  activities.  The  pilots  were  then 
asked  to  inspect  the  1  i  A-ll)  source  tasks  in  order  to  estimate  the  total 
workload  associated  with  each.  The  tasks  were  presented  in  the  format 
shown  in  Appendix  I.  However,  the  tasks  were  arranged  in  a  random  order  on 
.1  f r*  1  clout  pagi*  to  permit  simultaneous  examination  and  wore  identiliod  only 
by  the  letters  A-M. 
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In  order  to  reduce  the  difficulty  of  the  estimation  task,  the  pilots  were 
first  asked  to  study  the  communications  tasks  and  to  categorize  them  on  a 
five-point  workload  scale.  They  were  then  required  to  rank  order  all  13 
tasks  on  the  basis  of  workload. 

RESULTS 

Raw  ranked  data  such  as  those  obtained  for  this  study  do  not  reveal  the 
magnitude  of  the  underlying  differences  between  ordered  entities.  However, 
if  a  reasonable  assumption  about  the  relation  of  ranks  to  numerical  values 
can  be  made,  it  is  possible  to  convert  ranked  data  to  interval  scale 
values.  One  common  assumption  made  in  order  to  achieve  this  goal  is  that 
the  true  differences  between  adjacent  items  ranked  near  the  extremes  tend 
to  be  larger  than  differences  between  items  falling  near  the  middle  in 
rank.  Hays  (1069)  argued  that  when  multiple  judges  are  used  to  obtain 
rankings  this  assumption  becomes  reasonable  and  permits  the  development  ot 
a  scale  which  has  good  agreement  with  scales  derived  by  Thurstone's 
procedures. 

According  to  Hays  (I960),  in  order  to  derive  scale  values  from  ranks,  the 
relative  differences  between  N  items  are  viewed  as  being  similar  to  differ¬ 
ences  between  7.  values  falling  at  the  boundary  points  of  N-l  equally  prob¬ 
able  intervals  of  a  normal  distribution.  Thus,  the  interval  between  any 

two  adjacent  ranks  should  define  an  interval  corresponding  to  JiHl  percent 

N 

of  the  cases  in  the  distribution.  Furthermore,  in  order  to  place  the  scale 

values  in  the  midrange  of  the  distribution,  JiH  is  arbitrarily  set  as  the 

2N 

percentage  of  cases  below  the  value  of  the  item  ranked  1  and  above  the 
value  of  the  item  ranked  N.  These  assumptions  make  the  score  difference 
between  items  ranked  1  and  2  or  N  and  N-l  larger  than  the  difference 
between  items  ranked  in  the  center  of  the  range.  Interval  data  are  derived 
by  substituting  7,  values  for  the  ranks.  These  are  used  to  derive  mean 
scale  values  based  on  the  distribution  of  rankings  produced  by  several 
judges. 


The  procedures  described  above  were  applied  to  the  workload  rankings 
obtained  for  the  13  A-10  communications  tasks.  Table  11  summarizes  the 


data  and  the  scale  computations.  The  individual  cell  values  in  Table  11 
indicate  the  number  of  pilots  who  assigned  rank  x  to  task  v.  The  /.  score 


rank  fr)  equivalents  were  computed  by  finding  the  7.  score  cutting  off  the 
(  T*  *—  •  5  ) 

lower-1- — - — 1 proportion  of  the  area  under  the  normal  curve.  Thus,  for 

N  ( 1  -  . 5) 

rank  1,  the  lower  - -  or  .038  proportion  of  the  normal  curve  corres¬ 

ponds  to  a  7.  value  of  -1.77.  Scale  values  were  calculated  by  multiplying 
the  7.  value  for  a  rank  by  the  number  of  pilots  (f)  who  gave  that  rank  to 
the  task,  summing  across  ranks,  and  dividing  by  the  total  number  of  pilots 
(31). 


Thus,  for  Task  1  under  Identification  Demand  the  average  scale  value  was 
equal  to: 

T(Zf)  17C-1  .77)  +  4 (-1 .2)  +  8 (-.87)  +  1  (-.62)  +  1  ( .  19)  ,  ,,  , 

IT  - - Ti - -  -1  * 3(5,4 

To  eliminate  negative  scale  values,  the  final  scale  was  derived  by  adding 
the  absolute  value  of  the  largest  negative  average  value  to  each  of  the 
scores . 
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Section  5 


DISCUSSION 


SCALING  SUMMARY  AND  COMPARISONS 

The  primary  goal  of  the  effort  presented  in  this  report  was  to  derive 
standardized  communications  tasks  that  could  he  used  as  highly  realistic 
secondary  tasks  in  R&D  simulations  and  actual  flight  tests.  In  order  to 
combine  the  tasks  so  that  controlled  levels  of  loading  could  be  produced 
for  specific  applications,  three  methods  of  workload  scaling  were  employed. 
The  first  technique  was  based  on  estimating  the  information  transmission 
requirements  of  communications  activities.  Information  metrics  were  found 
to  be  applicable  to  perceptual  decisions  and  manual  action  decisions 
required  in  response  to  incoming  messages  ..nd  were  used  to  generate  an 
analytical  scale  for  the  13  A-10  communications  tasks.  This  approach  wa.-, 
supplemented  by  a  technique  developed  to  estimate  the  additional  contribu¬ 
tions  to  workload  of  information  gathering  activities,  memory  demands,  and 
instruction  complexity  which  were  not  assessed  bv  information  theoretical 
measures.  A  paired  comparisons  method  was  used  to  obtain  pilot  opinions  of 
the  workload  associated  with  individual  instructional  messages  contained  in 
the  communications  tasks.  The  results  were  used  to  produce  a  hybrid  ana¬ 
lytical  scale  representing  both  the  information  transmission  demands  and 
the  information  processing  complexity  of  the  tasks.  The  third  set  of 
workload  estimates  were  obtained  from  pilot  rankings  of  the  total  workload 
of  entire  communications  tasks.  Modified  Thurstonian  procedures  were  used 
to  devise  a  normalized  subjective  scale. 

Comparisons  of  the  results  of  the  three  scaling  methods  were  performed 
using  nonparametric  correlational  techniques.  Kendall's  Coefficient  of 
Concordance  revealed  a  significant  amount  of  overall  agreement  among  the 
workload  estimates  (W  =  .929,  p  .01).  :  nee  the  analytical  and  hybrid 

analytical  techniques  shared  data  sources  and,  therefore,  were  not  inde¬ 
pendent  estimates,  separate  Spearman  Rank  Correlations  were  also  computed 
between  the  individual  scales.  As  expected,  the  analytical  and  hybrid 
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analytical  scales  were  highly  correlated  (r  =  .070,  p  .01).  However, 

there  was  also  significant  agreement  between  the  subjective  scale  and  the 

findings  obtained  with  each  of  these  methods  (analytical,  r  =  ,83b, 

s 

p  ■  .01;  hybrid,  r^  =  .824,  p  ■  .01). 

WORKLOAD  THEORY  AND  SECONDARY  COMMUNICATIONS  TASKS 

Efforts  to  assess  mental  workload  using  secondary  tasks  are  complicated  bv 
a  somewhat  controversial  theoretical  climate.  Currently,  two  general  types 
of  models  of  the  human  information  processing  system  offer  different  recom¬ 
mendations  about  the  manner  in  which  secondary  tasks  should  be  implemented 
and  about  the  way  in  which  secondary  task  measures  can  be  interpreted  to 
make  inferences  about  workload  (see  Hawkins  and  Ketchum,  1080;  and  Sanders, 
1978  for  reviews  of  specific  models). 

In  one  type  of  theory,  mental  capacity  is  viewed  as  a  single  undifferen¬ 
tiated  resource  which  is  shared  among  information  processing  functions. 
Accordingly,  all  primary  and  secondary  tasks  should  draw  from  this  common 
capacity  and,  barring  peripheral  task  interference,  the  form  of  the  second¬ 
ary  task  should  not  bias  the  workload  measure  that  is  obtained.  The 
scaling  methods  used  in  this  project  to  estimate  the  workload  of  communi¬ 
cations  tasks  reflect  a  concept  of  workload  which  depends  upon  the  validity 
of  this  theory.  That  is,  the  communications  tasks  were  assigned  values  on 
unidimensional  scales  of  workload  that  did  not  differentiate  between  the 
information  processing  structures  or  functions  employed  by  the  task. 

A  second  type  of  workload  theory  assumes  that  the  information  processing 
system  has  several  structure-specific  capacities.  In  its  strong  form,  this 
model  might  be  used  to  argue  that  the  secondary  task  methodology  is  of 
limited  value  because  obtained  workload  measures  would  he  dependent  upon 
the  degree  to  which  particular  primary  and  secondary  tasks  share  common 
mental  functions,  and  therefore,  common  capacities.  However,  given  an 
appropriate  methodology,  a  multiple  resource  model  can  be  accommodated  hv 
the  secondary  task  technique.  As  Wickens  (1979)  has  noted,  if  meaningful 
differences  between  processing  functions  can  be  identified  and  if  secondary 
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tasks  to  assess  corresponding  loading  dimensions  can  be  selected,  the 
workload  of  a  primary  task  of  interest  could  be  expressed  as  a  structural 
loading  profile. 

At  present,  basic  research  efforts  have  failed  to  conclusively  demonstrate 
that  one  of  the  models  discussed  above  provides  a  better  description  of 
mental  workload  than  the  other.  Consequently,  it  is  impossible  to  deter¬ 
mine  whether  the  approach  taken  in  this  report  to  designing  secondary 
communicat ions  tasks  is  completely  adequate.  However,  should  the  contro¬ 
versy  be  resolved  at  a  future  date  in  favor  of  the  multiple  capacity  model, 
the  desirable  features  and  advantages  of  the  communications  task  method  of 
workload  assessment  could  be  incorporated  into  a  revised  methodology. 
Specifically,  it  is  entirely  feasible  that  subsidiary  radio  communications 
tasks  could  be  developed  to  tap  individual  processing  resources. 

FUTURE  RESEARCH 

The  development  of  the  concept  of  using  aircraft  radio  communications  tasks 
for  workload  assessment  and  the  use  of  a  priori  scaling  techniques  to 
estimate  the  workload  associated  with  these  tasks  are  initial  steps  toward 
designing  a  viable  workload  assessment  methodology.  A  program  of  dual  task 
performance  research  is  needed  to  assess  the  sensitivity  of  workload 
measurement  available  with  subsidiary  communications  tasks  and  to  generate 
a  criterion  for  validating  the  scaling  technique. 

Preliminary  studies  should  examine  the  performance  measures  which  can  be 
derived  from  the  verbal  and  manual  behaviors  associated  with  communications 
tasks  in  order  to  permit  the  selection  of  a  limited  set  of  indices  which 
appear  to  be  most  sensitive  to  primary  task  workload.  Once  measures  are 
obtained  for  each  task,  formal  research  must  be  conducted  in  which  the 
tasks  are  performed  by  subjects  in  conjunction  with  primary  aircrew  duties 
at  various  levels  of  task  demand. 

The  results  of  this  study  would  serve  two  purposes.  First,  the  obtained 
performance  scores  would  allow  an  ordering  of  the  tasks  along  a  dimension 
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ot  sensitivity  to  workload.  These  findings  could  then  he  used  to  determine 
the  type  and  amount  of  subsidiary  task  activity  needed  to  produce  useful 
workload  estimates.  Second,  the  scores  for  each  task  could  be  used  to 
assess  the  utility  of  the  a  priori  scaling  procedures.  Although  the  cor¬ 
relations  among  the  three  scaling  results  provide  evidence  for  convergent 
validity,  the  ultimate  criterion  for  evaluating  these  methods  is  the  task 
performance  variation  observed  in  a  dual  task  environment  such  as  that 
described  above.  Thus,  the  correlations  between  performance  scores  and 
each  of  the  scale  results  cou’d  be  used  to  select  an  appropriate  method  of 
constructing  communications  tasks  without  resorting  to  exhaustive  perfor¬ 
mance  testing. 

The  experimental  plan  outlined  above  could  be  implemented  in  an  austere 
flight  simulation.  Subjects  would  be  asked  to  perform  a  primary  tracking 
task  and  required  to  engage  in  simultaneous  radio  communications  tasks 
using  the  radio  panels  from  the  A- 10  aircraft.  The  results  of  this  pre¬ 
liminary  study  would  be  used  to  guide  future  efforts  with  the  radio  com¬ 
munications  task  workload  assessment  methodology.  Those  tasks  shown  to  he 
sensitive  to  rudimentary  continuous  control  task  workload  could  be  evalu¬ 
ated  at  a  higher  level  of  fidelity  of  simulation.  If  close  agreement  was 
found  between  dual  task  performance  and  the  results  of  any  of  the  a  priori 
scaling  techniques,  it  would  be  possible  to  write  generalized  guidelines 
tor  the  design  of  scaled  sets  of  additional  communications  tasks.  These 
procedures  would  then  lie  made  available  to  others  to  enable  the  development 
of  workload  measurement  tasks  tailored  to  specific  systems,  missions,  and 
crew  stations. 


ASK.b  (conilnutid) 
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APPENDIX  2 


A- 10  COMMUNICATION  CONTROLS 
AND  SWITCH  POSITIONS  AND  PROCEDURES  EOR  MISSION  SI  ART 


VI IF  I'M 
Manual  I'miv 


A- 10  il.'iwuu  i  at  ion  i\>nti'i«ls 


A-10  SWITCH  POSITIONS  AND  PROCEDURES  FOR  MISSION  START 


Radio  Panel 

Control 

Control  Position 

UHK 

Present  Channel  Card 

#1  marked  259.4 
#2  marked  328.2 

Channel  Selection 

1 

Frequency  Selectors 

178.3 

Mode 

PRESET 

Squelch 

ON 

Volume 

12  o'clock 

Function 

MAIN 

INTERCOM 

Volume 

12  o'clock 

Monitor  Switches 

UHK,  VII F,  FM  are  UP 
UHK,  FM  volumes  set 
higher  than  VHF 

HM,  INT,  UK,  IFF, 
ITS,  TCN  are  DOWN 

Rotary 

UIIF 

antenna  skeect 

IFF,  UIIF 

Both 

Radar 

Off 

VHP /AM 

Powe  r 

Power 

Frequency  Indicator 

122100 

Volume 

12  o'clock 

VHP /KM 

Sque 1 ch 

Ca r  r i e  r 

Frequency  Selectors 

4080 

Mode 

T/R 

IKE 

Ma  s  t  e  r 

Normal 

M-2 ,  M-b 

On 

M— 1 ,  M- 5/A 

Off 

Rad  Test 

(Inop) 

Id  ent 

Out 

Mode  5 /A 

0102 

Mode  1 

23 

Mode  4 

Out 

Audio 

Out 

Code 

A 

TRANSMIT 

Mic  Switch 

VHF  is  UP,  I'llF  is 

A— 1 0  SWITCH  POSITIONS  AND  PROCEDURES  FOR  MISSION  START  (continued) 


Radio  Panel 


THREAT 


UHK  Remote 

Frequency 

Indicator 


Control  Control  Position 

Engine  Restart  Left  switch  is 

CHAFF 

Right  switch  is 
FLARES 

(not  used  for  this 
report) 


TIC.ER's  radios  are  powered  ON  and  tuned,  and  his  INTERCOM  is  set  to  monitor 
UHK,  FM,  and  AM  and  transmit  on  UHK.  During  lost  or  jammed  communication, 
TIGER  is  to  try  alternate  frequencies  then  check  for  new  frequency  with 
POUNDER.  TIGER  should  always  return  to  UHK  (transmit)  unless  otherwise 
directed. 
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APPENDIX  3 


TASK  WORKLOAD  VALUKS 


Scenario 


No.  of  Bits 


I.  Identification  Demand 
A.  Task  1 


1 . 

Sender  ID 

2.585 

2. 

Receiver  ID 

2.585 

3. 

Push  IDENT  button 

1.000 

TOTAL  6.170 

Task 

2 

1. 

Sender  ID 

2.585 

2. 

Receiver  ID 

2.585 

3. 

IFF  Code  Select  Mode 

1.000 

4. 

IFF  Mode  3A/0400  Select 

12.000 

5. 

Push  IDENT  button 

1.000 

6 . 

Select  VH F /AM 

1.585 

7. 

Push  microphone 

1 .000 

8. 

Confirm 

1 .000 

TOTAL  22.755 

Tas  k 

3 

1. 

Sender  ID 

2.585 

2. 

Receiver  ID 

2.58  5 

3. 

Select  UHF  Preset 

Channel  5 

4.322 

4. 

Push  microphone 

1  .000 

5. 

Call  DOGBONE 

2.585 

6 . 

Report 

1 .000 

7. 

Select  UHF  Preset 

Channel  1 

4.322 

TOTAL  18.399 
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TASK  WORKLOAD  VALUES  (continued) 


Scenario 

No.  of  Bits 

u. 

Threat  Alert 

A.  Task 

1 

1. 

Sender  ID 

2.385 

2. 

Receiver  ID 

2.585 

3. 

Release  chaff 

1 .000 

4. 

Release  flares 

1 .000 

5. 

Select  FM 

1.585 

6. 

Push  microphone 

1.000 

7. 

Conf irm 

1 .000 

TOTAL  10.755 

B.  Task  2 


1  . 

Sender  II) 

2.585 

2 

Receiver  ID 

2.585 

3. 

Push  microphone 

1 .000 

4. 

Conf i rm 

1 .000 

5. 

Select  UHF  Preset 

Channel  5 

4.322 

h . 

Push  microphone 

1 . 000 

7. 

Call  DOG BONE 

2.585 

8. 

Report 

1.000 

9. 

Select  UIIF  Present 

Channel  1 

4.322 

TOTAL  20.399 


III.  Traffic  Control 

A.  Task  l 


L. 

Sender  II) 

2.585 

2  # 

Receiver  ID 

2.535 

3. 

Select  VHF/AM 

1.585 

30 


TASK  WORKLOAD  VALURS  (continued) 


Scenario  No.  of  hits 


A.  Task 

1  (continued) 

4. 

Rush  microphone 

1 .000 

5. 

Conf irm 

1 .000 

6. 

Select  DH K 

1.585 

TOTAL 

10.340 

15.  Task 

2 

1 . 

Sender  LI) 

2.585 

2 . 

Receiver  ID 

2.585 

3. 

Select  VHP 

1.585 

4. 

Push  microphone 

1 .000 

5. 

Conf irm 

1  .000 

6 . 

Select  VHP  9  (147.7)* 

11.966 

7. 

Push  microphone 

1 .000 

8. 

Report  to  POUNDKR 

2.585 

9. 

Select  UUP 

1.585 

TOTAL 

25.891 

Waypoint 

Passage 

A.  Task 

1 

1 . 

Sender  ID 

2.585 

2 . 

Receiver  ID 

2.58  5 

3. 

Push  microphone 

1  .000 

4. 

Confirm  (memory) 

1  .000 

5. 

Push  microphone 

1  .000 

b. 

Report  to  NAIL 

2.585 

TOTAL  10.755 


(*Assmne  POUNDKR  is  on  same  VHP  channel  and  there  are  no  presets) 
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TASK  WORKLOAD  VALUES  (continued) 


Scenario 

No .  of  B i t s 

B.  Task 

? 

1. 

Sender  ID 

2.585 

2. 

Receiver  ID 

2.585 

3. 

Push  microphone 

1 .000 

4. 

Confirm  (memory) 

1 .000 

5. 

Select  UUF  (132.1) 

12. 9bb 

6. 

Select  UHF  Manual 

1.585 

7. 

Push  microphone 

1  . 000 

8. 

Call  FRIENDLY  and  report 

2.585 

9. 

Select  UHF  Preset 

1.585 

10. 

Push  microphone 

1  .000 

11. 

Call  NAIL  and  report 

2.585 

TOTAL  30.47b 

V.  Jammed  Communications 

A.  Task 

1 

1. 

Sender  ID 

2.585 

2 . 

Receiver  ID 

2.585 

3. 

Change  channels 

1  .000 

(if  no  set  procedure) 

4. 

Push  microphone 

1  .000 

5. 

Call  NAIL,  check  for 

usable  frequency 

2.585 

TOTAL  9.755 

B.  Task 

2 

1 . 

Sender  ID 

2.585 

2. 

Receiver  ID 

2.58  5 

3. 

Change  channels 

1  .000 

4. 

Push  microphone 

1  .000 
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TASK  WUKK1.0A1)  VALliKS  (cont  inued) 


Scenario 

No.  of  Hits 

B.  Task 

3  (continued) 

s . 

Call  NALL  and  report 

2. 58 8 

b . 

Change  channels 

1  .000 

7. 

Push  microphone 

1  .000 

8. 

Call  POUNDKR, 

request  frequency 

2 . 388 

<■). 

Conf i ro 

1  .000 

10. 

Select  I'll K 

1.885 

11. 

Select  channel 

4.322 

i:;. 

Push  microphone 

1 .000 

n. 

Call  NAIL  and  report 

2.585 

TOTAL  24.832 

VI  . 

Strike  Clearance 

A.  Task 

1 

1. 

Sender  ID 

2.585 

') 

Receiver  ID 

2.585 

3. 

Push  microphone 

1  .000 

4. 

Confirm  (memory) 

1.000 

5. 

Push  microphone 

1  .000 

b . 

Call  NAIL  and  report 

IP  (memory) 

2.585 

7. 

Push  microphone 

1.000 

8. 

Call  NAIL  and  report  Pop-up 

2.585 

TOTAL  14.340 

15.  Task 

') 

1  . 

Sender  ID 

2.585 

2 . 

Receiver  ID 

2.58  5 

3. 

Push  microphone 

1  .000 

4. 

Confirm  (memory) 

1  .000 
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TASK  WORKLOAD  VALUES  (continued) 


Scenario 

No.  of  Bits 

B.  Task 

2  (continued) 

5. 

Push  microphone 

1 .000 

6. 

Call  NAIL  and  report 

IP  (memory) 

2.585 

7. 

Push  microphone 

1 .000 

8. 

Call  NAIL  and  report  Pop-up 
(memory) 

2.585 

9. 

Push  microphone 

1 .000 

10. 

Call  NAIL  and  report 

Target  Recognition  (memory) 

2.585 

11. 

Select  VHP/FM 

1.585 

12. 

Push  microphone 

1 .000 

13. 

Call  Tiger  2  and  report  Clear 

2.585 

14. 

Select  UHK 

1.585 

TOTAL  24.680 


APPKNDIX  4 

INSTRUCTIONS  AND  SAMP1.K  PACK  FROM 
PAIKK1)  COMPARISONS 

TKST  BOOKLKT  INSTRUCTIONS 

The  purpose  of  this  study  is  to  get  pilot  opinions  on  the  workload 
associated  with  tasks  that  you  are  required  to  perform  in  response  to 
particular  radio  messages.  You  will  be  asked  to  compare  a  number  of 
messages  containing  instructions  that  you  might  receive  during  an  air- 
to-ground  attack  mission.  The  comparisons  will  be  performed  in  pairs. 

When  evaluating  the  instructions,  you  should  try  to  consider  all  of  the 
visual,  mental,  manual,  and  verbal  activities  that  you  would  have  to  engage 
in  and  their  effects  on  the  workload  you  would  experience.  Although  the 
individual  instructions  may  be  taken  from  particular  mission  segments  which 
differ  in  the  total  amount  of  workload  associated  with  them,  we  want  you  to 
try  to  evaluate  only  the  workload  imposed  upon  you  by  the  specific  instruc¬ 
tion.  Try  not  to  confuse  the  workload  level  of  the  primary  mission  or  the 
combat  situation  with  your  perception  of  the  workload  due  to  the  instruc¬ 
tions  received  over  the  radio. 

For  each  pair,  we  want  you  to  place  a  check  next  to  the  instruction  which 
requires  you  to  invest  the  greater  amount  of  effort  to  carry  out  (i.e., 
which  has  the  higher  workload).  DO  THIS  NOW. 


SENDER 


MESSAGE 


9-12 


10  -  1J 


11  -  14 


12  -  n 


FAC  Call  in  hot  at  POP. 


AWACS  Squawk  I  DENT. 


FAC  Ca 11  t  a  rge t  ins igh t . 


AWACS  Squawk  3,  0400. 


FAC  Cal  1  cl  oar  to  TIGER  2. 


FAC  Go  to  CUE  b 


AWACS  Squawk  I  DENT. 


TIGER  2  Break  loft,  SAM  at  A  o’clock 


AWACS  Report  SAMS 


1  -  b 


AWACS 


Descend  to  base  plus  3,  turn  90  degrees  right, 
hold  for  i  minute.  Report  at  altitude. 
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